15 research outputs found
Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction
We tackle image question answering (ImageQA) problem by learning a
convolutional neural network (CNN) with a dynamic parameter layer whose weights
are determined adaptively based on questions. For the adaptive parameter
prediction, we employ a separate parameter prediction network, which consists
of gated recurrent unit (GRU) taking a question as its input and a
fully-connected layer generating a set of candidate weights as its output.
However, it is challenging to construct a parameter prediction network for a
large number of parameters in the fully-connected dynamic parameter layer of
the CNN. We reduce the complexity of this problem by incorporating a hashing
technique, where the candidate weights given by the parameter prediction
network are selected using a predefined hash function to determine individual
weights in the dynamic parameter layer. The proposed network---joint network
with the CNN for ImageQA and the parameter prediction network---is trained
end-to-end through back-propagation, where its weights are initialized using a
pre-trained CNN and GRU. The proposed algorithm illustrates the
state-of-the-art performance on all available public ImageQA benchmarks
Real-Time Object Tracking via Meta-Learning: Efficient Model Adaptation and One-Shot Channel Pruning
We propose a novel meta-learning framework for real-time object tracking with
efficient model adaptation and channel pruning. Given an object tracker, our
framework learns to fine-tune its model parameters in only a few iterations of
gradient-descent during tracking while pruning its network channels using the
target ground-truth at the first frame. Such a learning problem is formulated
as a meta-learning task, where a meta-tracker is trained by updating its
meta-parameters for initial weights, learning rates, and pruning masks through
carefully designed tracking simulations. The integrated meta-tracker greatly
improves tracking performance by accelerating the convergence of online
learning and reducing the cost of feature computation. Experimental evaluation
on the standard datasets demonstrates its outstanding accuracy and speed
compared to the state-of-the-art methods.Comment: 9 pages, 5 figures, AAAI 2020 accepte
Retrospective Loss: Looking Back to Improve Training of Deep Neural Networks
Deep neural networks (DNNs) are powerful learning machines that have enabled
breakthroughs in several domains. In this work, we introduce a new
retrospective loss to improve the training of deep neural network models by
utilizing the prior experience available in past model states during training.
Minimizing the retrospective loss, along with the task-specific loss, pushes
the parameter state at the current training step towards the optimal parameter
state while pulling it away from the parameter state at a previous training
step. Although a simple idea, we analyze the method as well as to conduct
comprehensive sets of experiments across domains - images, speech, text, and
graphs - to show that the proposed loss results in improved performance across
input domains, tasks, and architectures.Comment: Accepted at KDD 2020; The first two authors contributed equall